Running head: TRAJECTORY FORMS AND EVENT RECOGNITION (P525) Trajectory Forms as Information for Visual Event Recognition: 3D Perspectives on Path Shape and Speed Profile

Trajectory forms in events consist of the path shape and the speed profile (Bingham, 1987; 1995). Wickelgren and Bingham (2004) showed that adults can use the speed profile as visual information to recognize events from different perspectives, despite perspective distortions and differences in optical components. We now investigate whether adults can use trajectory forms to recognize events when the path shape varies as well as the speed profile and the forms are viewed from 3D perspectives. In Experiment 1, we tested recognition of events that differ in path shape (with the speed profile held constant). In Experiment 2, we tested recognition of events in which speed profiles were mapped onto circular paths. In Experiment 3, as a strong test of sensitivity to trajectory forms, we tested simultaneous separate recognition of speed profile and path shape when both varied across events. In all 3 experiments, events were viewed from multiple perspectives in 3D. The results show that both the shape of the path and the speed profile provide information for visual event recognition. We found that adults exhibit constancy (or view invariance) in being able to use trajectory forms to identify the same events when viewed from different perspectives in 3D. Trajectory Forms and Event Recognition (P525) 3 Trajectory Forms as Information for Visual Event Recognition: 3D Perspectives on Path Shape and Speed Profile Visual event perception is the process and ability of identifying events based on the information in the optic flow pattern. The earliest studies of information in visual event perception focused on the phase relations among the visible points in rhythmic or oscillatory events (Johansson, 1950). The role of relative phase has been studied extensively both in visual event perception studies (Bingham, 1995; Bertenthal,1996; Bertenthal & Pinto, 1993; 1994; Bertenthal Proffitt & Cutting, 1984; Bertenthal, Proffitt & Kramer, 1987; Booth, Pinto & Bertenthal, 2002) and in studies on perception in bimanual coordination (Bingham, 2004; Bingham, Schmidt & Zaal,1999; Bingham, Zaal, Shull & Collins, 2001; Wilson & Bingham, 2005a; 2005b; Zaal, Bingham & Schmidt, 2000). More generally, however, events can be understood as spatial-temporal objects that exhibit characteristic shapes or forms just as do more familiar spatial objects (Bingham, 1987; Bingham, 1995; Bingham, Rosenblum & Schmidt, 1995: Runeson, 1974). Bingham (1995) formulated this in terms of trajectory forms. A trajectory consists of variations in position and speed of a moving point. Trajectories exhibit path shapes and speed profiles along those paths. Such trajectory forms are different in different events. The oscillatory trajectory of a bouncing ball is quite different from that, for instance, of a manually oscillated hammer being used to hammer in a nail (Bingham, Rosenblum & Schmidt., 1995). If trajectory forms are event specific, then trajectory forms might be used to recognize events just as the shapes of objects can be used to recognize them. Runeson (1977) argued that the physical dynamics responsible for generating the motions in an event produce unique kinematic (or motion) properties that enable observers who detect Trajectory Forms and Event Recognition (P525) 4 those motions to perceive corresponding dynamic properties of the events. Runeson developed this idea, which he called “KSD” (Kinematic Specification of Dynamics), in the context of his work on perception of the mass of objects in collision events or amounts of weight in a human lifting event (Runeson, 1977; Runeson & Frykholm, 1981; 1983). These are examples of the scaling problem in visual perception. How can the spatial-temporal optical pattern, which is angular (and therefore, only time dimensioned) provide information about scale properties in events other than time (e.g. distance, size, mass, etc.)? Subsequent studies showed that the timing of readily identified gravitationally governed events (pendular events, balls rolling downhill, bouncing balls, falling water, walking dogs, etc.) provides information allowing the size and distance of objects in the events to be judged (Jokisch & Troje, 2003; McConnell, Muchisky & Bingham, 1998; Pittenger, 1985; 1990; Stappers & Waller, 1993; Twardy and Bingham, 2002; Watson, Banks, von Hafsten & Royden, 1992). The solution revealed by these studies is that the trajectories of visible points in an event are lawfully governed by the underlying event dynamics, which uniquely couple spatial and temporal properties so that one can provide information about the other. Bingham (1995) extended Runeson’s treatment to address the problem of event recognition. Bingham et al. (1995) showed that simple events consisting of a single moving point could be distinguished and recognized using the trajectory forms. Observers discriminated a freely bouncing object from one moved by hand. A freely swinging object was discriminated from one moved by hand. Furthermore, each event was recognized and, in particular, whether events were animate or inanimate was correctly identified. Other more complex events were similarly recognized (for instance, wind blown objects, objects moving in liquid that was stirred or splashed, a kicked ball, a ball rolling downhill). Furthermore, the trajectory forms of many of Trajectory Forms and Event Recognition (P525) 5 these events were asymmetric and orientation specific with respect to gravity. Bingham et al. found that when such trajectory forms were inverted, the events were no longer recognized. Similarly, observers have failed to recognize human walking when the event kinematics were inverted (Pavlova & Sokolov, 2000; Sumi, 1984). Bingham et al. (1995) also tested the effect of changes in the relative orientation of observers and events by having inverted observers judge upright event kinematics. In this case, the events were correctly identified. These results show that the events are perceived relative to the generative dynamical context, that is, the downward gravitational force. The trajectory forms of events are specific to the dynamics that deterministically generate them. If the dynamics are specific to the type of event, then trajectory forms can provide information about event types that would allow them to be recognized. Bingham et al. provided evidence that events are perceptually taxonimized in terms of the types of underlying generative dynamics. These studies provided some evidence that human observers are sensitive to trajectory forms and are able to use them to recognize events. A number of questions remain. First, trajectory forms can vary in two ways. One is that the form of the speed profile along a given path of motion can vary. The other is that the form of the path of motion can vary for a given speed profile. Are observers sensitive to both dimensions of variation of trajectory forms? Second, trajectory forms are properties of events that must be projected into optic flow to be detected by the visual system. What are the optic flow variables and how are the two aspects of trajectory forms, path and speed, mapped into those variables? Third, the trajectory forms of events map into spatial-temporal optical patterns via perspective projections and thus, the forms are subject to perspective distortions just as are the shapes of objects when projected into optical images. Does event recognition exhibit constancy (or view invariance), that is, does a given Trajectory Forms and Event Recognition (P525) 6 trajectory form allow the event to be recognized correctly when it is viewed from different 3D perspectives? Previous studies have addressed some of these questions. Muchisky & Bingham (2002) showed that adults can use the information in the speed profile of a nonlinear oscillator to recognize the event. An object oscillated along a straight path in a frontoparallel plane. Thresholds for both asymmetric (skew) and symmetric (kurtosis) variations in form were measured as well as the ability to use the corresponding forms to reliably recognize the corresponding events. Thresholds were found to be comparable to those established for velocity discrimination and observers were able to use the forms to recognize events. Wickelgren and Bingham (2004) showed that people can use the information present in the speed profile of an event to recognize the same event when viewed from different perspectives. Participants viewed an object oscillating along a straight path in a frontoparallel plane with one of five different speed profiles and identified which event they were viewing. Then, the same events were viewed from a perspective looking along the straight path extending away in depth so that the object moved towards and away from the observer. The perspective change incurred large perspective distortions in the trajectory forms. At the same time, the optical variables changed from rigid image translation to nonrigid image expansion and contraction. (Note: Johansson (1950) referred to these as common and relative motion components, respectively.) Participants continued to be able to recognize the events despite these changes. When the path of motion was viewed from an oblique angle, the perspective distortions were carried simultaneously by both optical variables, that is, image translation and expansion/contraction and the events were still recognized. Thus, human observers are able to detect speed profiles and use them to recognize events despite projective distortions that can Trajectory Forms and Event Recognition (P525) 7 occur with changes in 3D perspective. Trajectory forms also vary in respect to path shape. The current studies were designed to investigate the use of path shape as information about event identity as well as the combination of path shape and speed profile. In Experiment 1, different events were created by varying the shape of the path along which an object traveled at constant speed. Observers became familiar with the events by viewing them from a perspective, which placed the paths in a frontoparallel plane. Subsequently, recognition was tested when events were viewed with the motion in a horizontal plane at eye level (thus, extending in depth) to determine the potential effects of perspective distortions. In Experiment 2, an object moved along a circular path with different speed profiles. Again, observers became familiar with the different events while viewing motion in a frontoparallel plane. Subsequently, recognition was tested while viewing motions in depth. Finally, in Experiment 3, both path shapes and speed profiles were varied. Five different path shapes were combined with five different speed profiles to create 25 different events. The ability to recognize the events in terms of the path shapes and speed profiles was tested when observers viewed the motions in depth so that viewing was subject to perspective distortions. As shown in Appendix A, the speed profile and path shape of a trajectory form are specified in the optical flows. If To is the optical translation and Eo is the optical expansion/contraction component, then the speed profile is specified by V(t) = D(t) * sqrt(To (t) + Eo (t)), where D(t) is the viewing distance. If viewing distance is large (that is, D >> ∆D, where ∆D = DmaxDmin over the course of the event), then D(t) ≈ Dc (that is, D constant). At closer distances, D(t) = Dc yields perspective distortions as discussed and investigated by Wickelgren and Bingham (2004). The path shape requires the direction of motion, Ø(t), in addition to the speed. Trajectory Forms and Event Recognition (P525) 8 The direction is specified by Ø(t) = arctan[Eo(t)/To(t)]. Given the availability of both speed and path in the optics, we expected that observers might be able to perceive these events in terms of these two separable aspects of the form rather than as a single integral form. That is, they should be able to distinguish a set of speed profiles and a set of path shapes rather than the set of events composed as the product of these two sets. EXPERIMENT 1: RECOGNITION OF PATH SHAPE FROM DIFFERENT PERSPECTIVES Experiment 1 was designed to test how well people can recognize different path shapes when viewed from different perspectives. The speed profile was held constant and was, in fact, a constant speed. Observers became familiar with 5 different path shapes viewed in a frontoparallel plane. Then, they were asked to identify these shapes when viewed in depth, that is, the plane containing the path was horizontal at eye level and thus, viewed edge on.